Information retrieval systems search in a database to retrieve information the user is looking for. In information retrieval systems the biggest challenge is to understand the user's needs. A system based on keyword search often has a limitation in the query language and indexation of the database. For the user it is difficult to express accurately in keywords what he is looking for. Information retrieval in an image based database is even more difficult, because it is difficult for systems to interpret an image in the same manner as a human being. A successful paradigm to solve this problem has been “relevance feedback”. Such a system retrieves candidate data objects that match the search query, the user reviews the retrieved candidate data objects and provides user feedback on the relevance or irrelevance of the candidate data objects. The system learns from the user feedback to improve its search performance. If a candidate data object is similar to a particular data object for which the user is searching, the user selects the candidate data object as being relevant for the search. If a candidate data object is not similar to the user's intensions or even a completely wrong search result, the user selects the data object as being irrelevant for the search. Thus, the system learns from the feedback that it has to search for data objects that are similar to the relevant candidate data objects and that are not similar to the irrelevant candidate data objects.
A system for image retrieval is described in an article by Qi Tian, et al, entitled “Combine user defined region of interest and spatial layout for image retrieval”, in Proc. of IEEE 2000 International Conference on Image Processing (ICIP'2000), pp. 746-749, Vol. 3, Vancouver, BC, Canada, Sep. 10-13, 2000. The system disclosed in the reference document tries to find image objects in a database of image objects based on a query image. The system has to find images that are similar to the query image. The images in the database are subdivided in a grid of pre-defined regions. For every region a feature vector is present in the database. The system compares the query image with the images in the database to find images that have feature vectors for regions similar to the feature vectors of the regions of the query image. The images in the data with most similar feature vectors are presented to the user as return images. Return images are candidates for the image, or images, the user is looking for.
Subsequently the user has to provide relevance feedback on a number of return images such that the system learns about the intentions of the user. This feedback triggers an additional search wherein the system tries to find images that better match the intentions of the user. After a number of iterations a set of images is found by the system best matching the intentions of the user. In order to allow the system to optimally learn the intentions of the user, the system requires feedback on many return images and thus many iterations are required.
In addition, the system offers the user the possibility to define together with the query image a Region Of Interest (ROI). The user indicates to the system which part of the image has his interest and indicates as such that the system has to find similarities between the query image and the image in the database inside the ROI only. Information in the query image outside the ROI is not of interest to the user and needs to be ignored by the system. The system uses the ROI to determine weight values for the regions of the grid of regions. Regions that fall completely outside the ROI get a weight value of 0. Regions that partly overlap with the ROI get a lower weight value than the regions that completely overlap with the ROI. Regions with a low weight value will only marginally be taken into account in the search, and regions with a high weight value will influence the search results most. Each iteration the system uses the same set of weight values for the searches. Although the focus of the search is marginally better, the user has still to provide feedback on many images.
Reviewing many return images to provide feedback regarding the return image and going through many iterations is especially a disadvantage in the medical domain where the users of the systems are medical experts who have limited time and resources. Furthermore, in the case that the images need to be transmitted to a handheld device for receiving feedback, for example, to a handheld device that the medical expert uses when visiting patients, the number of wireless transmissions must be minimized because of limited available bandwidth for the handheld device.